{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural language inference \n",
    "In the Natural Language Inference (NLI) task, the goal of our model is to determine whether a hypothesis is an entailment (true), contradiction (false), or undetermined (neutral) given a premise. Let us learn how to perform natural language inference task by finetuning the BERT.\n",
    "\n",
    "Consider the sample dataset shown in the following, as we can observe, we have a premise and hypothesis with a label indicating whether they are entailment, contradiction, or undetermined.\n",
    "\n",
    "\n",
    "![title](images/7.png)\n",
    "\n",
    "Now the goal of our model is to determine whether the sentence pair (premise-hypothesis pair) is an entailment, contradiction, or undetermined. Let us understand how to do this with an example. Consider the following premise-hypothesis pair: \n",
    "\n",
    "Premise: He is playing\n",
    "\n",
    "Hypothesis: He is sleeping \n",
    "\n",
    "First, we tokenize the sentence pair, add [CLS] token at the beginning of the first sentence and [SEP] token at the end of every sentence. The tokens are shown below: \n",
    "\n",
    "tokens = [ [CLS], He, is, playing, [SEP], He, is, sleeping [SEP]]\n",
    "\n",
    "Now, we feed the tokens to the pre-trained BERT and get the embedding of each token. We learned that the representation of the [CLS] token holds the aggregate representation.\n",
    "\n",
    "So, we take the representation of the [CLS] token which is $R_{[CLS]}$ and feed it to a classifier (feeedforward+softmax) which returns the probability of a sentence to be a contradiction, entailment, and neutral. Our results will not be accurate in the initial iteration, but over a course of iterations, we will have a better result. \n",
    "\n",
    "\n",
    "![title](images/8.png)\n",
    "\n",
    "Now that we learned how to finetune BERT for the natural language inference task, in the next section, we learn how to finetune BERT for the question-answering task. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
